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INTRODUCTION 

This paper describes an imaging system which applies knowledge- 
based technology to supervise and control both sensor hardware and 
computation in the imaging system. It includes the development of an 
imaging system breadboard which brings together into one system work 
that we and others have pursued for LaRC for several years. 

(Refs 1,2,3). The goal is to combine Digital Signal Processing 
(DSP) with Knowledge-Based Processing and also include Neural Net 
processing. 

The system is considered a smart camera. Imagine that there is 
a microgravity experiment on-board Space Station Freedom with a high 
frame rate, high resolution camera. All the data cannot possibly be 
acquired from a laboratory on Earth. In fact, only a small fraction 
of the data will be received. Again, imagine being responsible for 
some experiments on Mars with the Mars Rover: the data rate is a few 

kilobits per second for data from several sensors and instruments. 
Would it not be preferable to have a smart system which would have 
some human knowledge and yet follow some instructions and attempt 
to make the best use of the limited bandwidth for transmission? 

This paper will describe the system concept, current status of 
the breadboard system and some recent experiments at the Mars-like 
Amboy Lava Fields in California. 


SYSTEM OVERVIEW 


The system architecture concept is shown in Figure 1. The four 
areas shown are sensors, focal plane processor, knowledge-based 
supervisor/controller and image processors. Inputs to the system are 
supervisory commands, channel capacity and other mission data. The 
output is edited, classified and coded data, as well as other 
feature and range information. 

Internal communications are provided so that sensors and 
processing can be supervised and controlled by the knowledge-based 
system. Rules, knowledge, data and researcher's expertise are 
contained in the Knowledge-Based Supervisor/Controller KBSC. 

tThis work was supported by the following NASA contracts: 

NAS1-18816 and NAS1-18664. 
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The data from multiple sensors are simultaneously processed and 
combined by the KBSC. 

Signal and image processing will be performed both in the focal 
plane processor and with the image processor. 

The parallel asynchronous processor and the KBSC and image 
processing hardware are described in other papers (Refs. 2, 3). The 
laser ranger system and goals of the KBSC system will be discussed 
here. 


Figure 2 is a different view of the knowledge-based image/ 
sensor fusion breadboard in that it segments the knowledge-based and 
signal processing in the manner in which it is implemented in the 
breadboard system. The knowledge base is hosted on a SUN computer 
and the real-time signal/video processing is on the Datacube 
pipeline image processor. The sensors are CCD cameras, IR sensors 
and laser ranger. 


INTEGRATED LASER CAMERA 


The integrated laser camera combines range data from the laser 
with the spatial reflectance data from the CCD camera. The concept 
is to provide a sensor which has both high spatial resolution and 
accurate range to specific objects in the scene. Table 1 summarizes 
the limitations of either sensor separately and the potential of 
fusing the data from each to give both range and spatial detail. 

Table 2 lists the function of the ILC. The range may be 
provided at a single point or a range image may be generated by 
scanning the ranger over an area. Several display functions are 
available such as a contour map and an artificial grid to provide 
the concept of depth and range to any object in the scene. Camera 
control functions such as focus and zoom can be performed from the 
range output. Combining the range with high frequency spatial data 
can achieve rapid and very reliable camera focus. 

Combining range and spatial data has many applications in image 
analysis, such as segmentation, distinguishing objects from shadows, 
obstacle avoidance, etc. 


LASER RANGER 


There are basically two types of lasers, pulse and continuous 
wave. Pulse laser rangers operate on the basis of measuring the time 
it takes a laser pulse to travel to the object and back to the 
receiver. A continuous wave (CW) laser ranger normally compares the 
phase shift between the transmitted and received wave. A discussion 
of the merits of CW vs. pulse is beyond the scope of this paper 
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except to say that pulse laser rangers require less computation for 
range and generally are better for long-range imaging, for example, 
beyond 200 meters. Range ambiguity also needs to be resolved when 
using a CW laser ranger. 

We selected a pulse laser shown in Figure 3 with the 
specifications shown in Table 3. Range, accuracy and firing rate 
were all important in selecting the laser ranger. The 2000 firings 
per second makes it quite feasible to generate small range images. 

An accuracy of about 5cm to 10cm can be achieved by averaging over a 
number of firings on a single object. 

Figure 4 shows the block diagram implementation of the 
integrated laser/CCD prototype system. The laser ranger is mounted 
on a precision pan/tilt platform. The platform is controlled by the 
SUN either to point to a specific object or to scan an area to 
generate a range image. The camera focus and zoom will be controlled 
by the laser ranger in the breadboard imaging system. The video rate 
Datacube processor generates a graphics overlay on the video image 
such as the laser beam location. 

Figure 5 shows the platform design for the ILC. The azimuth 
motion is provided with a rotary stage platform on which is mounted a 
goniometer for the elevation motion. The platform has a repeat- 
ability of about 0.17mR about each axis. The laser ranger will be 
bore sighted with the camera axis. 


KNOWLEDGE -BAS ED SUPERVISOR/CONTROLLER 

The objective of the KBSC is to provide a robust, flexible 
monitoring and control system which provides sensor control, 
processing and image coding control, outputs edited, classified and 
coded data based on an internal knowledge-base and data base, sensor 
input, and supervisory command. 

Figure 6 shows some of the input, outputs and control functions 
of the KBSC system. The inputs to the KBSC can be from image 
processors such as the spatial frequency, histogram or other computed 
characteristics of the image. It may be a previously identified 
object or area so as to designate a small region of interest (ROI) . 
Edge information and segmentation may be used to identify specific 
features in the image. The color or more generally the spectral 
response of the image may be used to identify regions or objects. 

From the laser ranger, range and reflectance data may be used 
with the spatial data to identify features. Laser reflectance values 
may be used to determine the reliability of the range data and to 
determine the reflectance of the target at the laser frequency. The 
field of view (FOV) may be important when selecting processing 
algorithms. Sun angle, available bandwidth, and other priorities 
will be used to select processing algorithms and image coding 
methods. 
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In addition to sensor control such as focus, the KBSC will also 
be useful in applying the user's knowledge to select the information 
and data which is important. 


AMBOY DATA COLLECTION 


Some of the features of the breadboard imaging system are 
implemented in the Odetics Mobile Imaging Laboratory. Figures 7, 8, 
and 9 show the Mobile Imaging Laboratory platform with infrared and 
CCD cameras and platform with the laser ranger. Figures 10 thru 16 
are samples of a set of images taken with the Mobile Imaging Lab at 
the Amboy Lava Field in the Mohave Desert which looks very much like 
Viking pictures of Mars. Figures 10 and 11 are CCD and laser scan 
images respectively of approximately the same area. Some of the very 
black lava in the lower right band corner did not reflect enough 
signal for the range computation. The bush in the center of the 
scene is clearly visible. (The black and white print was made from a 
pseudo color range image (Figure 11) , causing the ranges at 50-60 
meters to appear closer.) 

Figures 12 and 13 are photographs of Amboy terrain which looks 
remarkably like the Mars Viking pictures except for a few bushes. 

Figures 14 and 15 are infrared images (8-12/m) of the Amboy 
crater taken when the temperature was about 110 °F. The white areas 
in the IR image are hot. 

Figure 16 shows a panoramic scene of the Amboy Lava Field taken 
with a series of images with a narrow f ield-of-view camera on the 
pan/tilt platform. Figure 17 is an artist conception of how a Mars 
Rover camera might assemble narrow f ield-of-view pictures into a 
panorama. The image was digitized from an actual Mars Viking 
photograph. 
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TABLE 1 


INTEGRATED LASER RANGER/CAMERA SYSTEM 

Problem: 

Existing robot vision systems do not provide both range and high 
resolution. 

■ Extracting range from the image data of CCD cameras is 
extremely computer intensive and provides poor 3-D contour data. 

■ Laser ranging/scanning systems provide range and 3-D contour 
data, but image detail is poor. 

Solution: 

Provide intelligent fusion of high resolution CCD camera data and 
laser ranging data. 

Use an expert system for determining how the data from different 
sources will be used to extract scene data. 

TABLE 2 

ILC FUNCTIONS 

• Range to any point in scene 

• Range image of any area in scene 

• Display (range image, contour map) 

• Display depth grid 

• Display range to any point 

• Focus camera 

• Combine range and reflectance data 
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TABLE 3 


501 LASER RANGER SPECIFICATIONS 


Range 

1 0-500 m 

Accuracy 

.2m 

Resolution 

.1m 

Beam divergence 

2.5 mR 

Measurement rate 

1-2000/sec 

Weight 

3 Kg 

Power 

3A@ 12V 
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Figure 1 Knowledge-Based Image/Sensor Fusion System 



Figure 2 Knowledge-Based Imaging/Sensor Fusion System 
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Figure 3 Rangefinder 501/SX 



COLOR MONITOR 


Figure 4 


ILC System Block Diagram 













CCD 

CAMERA 



Figure 5 Platform Design for the ILC 



Figure 6 


Knowledge-Based Supervisor/Controller 









Odetics Mobile Imaging Laboratory 


Sensor Platform with Infrared Camera 
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Figure 9 


Sensor Platform with Laser Ranger 



Figure 10 CCD Image Amboy Lava Field 
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Figure 11 Laser Ranger Image of CCD Image (Figure 10) 



Figure 12 Photograph of Mars-Like Amboy Lava Field 
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Figure 15 


Photograph of the same areas as Infrared Image, 
(Figure 14) . 



Figure 16 Panorama of Narrow Field-of-View Image of Amboy Lava 

Field 
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Figure 17 Artist's Conception of Mars Panorama Using Actual 

Viking Mars Image. 
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